Machine vision systems are used for a growing variety of manufacturing and inspection processes in research and industry. A machine vision system enables a user to monitor subject parts and workpieces, either at stationary locations or on moving lines, and to compare acquired images of the subjects to trained patterns, stored in a memory.
FIG. 1 shows a typical arrangement for a basic machine vision system 100 according to the prior art. An image sensor unit 102 is directed at an object (exemplary gear 104) within the viewing area 108 of the image sensor 102. The sensor includes a lens 110 of appropriate viewing angle, aperture and magnification. These parameters can be varied, either by remote control, automatic adjustment or manually. The lens transmits viewed light to the electro-optical pickup, which can be a charge couple device (CCD) 112. Basic interfaces (not shown) can be included, which interact with the sensor circuitry—such as triggers and strobes. Such interfaces are used to direct the capture of images, operate shutters, illumination and carry out other basic image acquisition functions. Likewise, integral or remote illumination 114 can be linked to the sensor, and operate during image acquisition, power-on or upon occurrence of another desired event. This illumination is directed at the viewing area 108 in this example. The viewing area 108 can, itself, include a backlight for contrasting the object 104.
The sensor unit 102 is connected by cabling 120 to an image processor/machine vision card 122. This card can be connected via a bus (such as a Peripheral Component Interface (PCI) bus) 123 to an appropriate bus slot of the personal computer (PC) or workstation 124. It can reside within, or remote from, the computer chassis. The machine vision card 122 is a specialized piece of hardware and firmware that includes a framegrabber, data sampling, compression/decompression algorithms and other functions needed to effect rapid capture and conversion of image data from the sensor into bus-based, computer-readable data. It may also include various control interfaces for transmitting computer-generated instructions back to the sensor to control various imaging functions, as well as trigger, strobe and illumination functions.
The computer 124 is typically configured with a conventional operating system, such as Windows® NT, from Microsoft Corporation of Redmond, Wash., open source Linux, or another similar operating system, which supports a display 126 with a graphical user interface (GUI). The GUI is manipulated by a keyboard 128, mouse 130 and other appropriate user interfaces (such as a touchscreen). The display can be used to show real time and stored images 132 of the object 108, and text or GUI control buttons by which the image can be analyzed or manipulated. A variety of other machine vision functions and tools can also be provided, all of which are part of a generalized machine vision application 140, stored on disk, and loaded as needed by the system. In one example, these tools allow a user to train the system to recognize certain image parameters, act upon those parameters and deliver test data and/or decision functions to the user. One such decision function may be whether to operate a robot, manipulator or conveyor with respect to the viewed object to accept or reject the object. An appropriate interface to a robot controller or other responsive mechanism 142 can be provided. Likewise, the mechanism 142 can deliver signals to the computer (for example a conveyor/part advance signal) that instructs capture of an image or other related operation at the sensor. Note that, in the alternative, the vision system can be implemented as a basically stand-alone unit that does not utilize a separate PC with a full operating system and file system to carry out the above-described vision system functions. In this case, the unit could include a purpose-built operating system and hardware to carry out the machine vision function, display and any associated operator interface function.
The computer 124 may be connected to a number of sensor units and/or inspection areas throughout a work area. Similarly, the computers may be networked via a conventional network interface to a main server and/or central management computer. In general, the PC has become the preferred choice for providing a human/machine interface (HMI) because of it's versatility, relatively low cost, standardization, wide availability, ease of use and familiarity for a wide range of users. The typical HMI allows the user to perform a variety of interactive functions with the machine vision system (MVS). In particular, machine vision HMI requires five general categories of operations that must be performed by users (e.g. installers, technicians, engineers, operators, and/or supervisors):
Installation—A machine vision sensor must be mounted at an appropriate position and attitude in space so that it sees the desired field of view. The lens must be focused, and its aperture must be set to admit an appropriate amount of light. Additionally, an object or pattern of known size and shape is typically used to calibrate the sensor to a real-world coordinate system. MVS installation has traditionally required a mechanism for the human installer to see live, real time images generated by the sensor.
Configuration—A user chooses appropriate operating parameters that specify such things as how images are to be acquired, the machine vision operations to be performed, the acceptance criteria, and how the results are to be communicated. Configuration requires a mechanism for a user to enter numeric, Boolean, enumerated, text string, and other parameters, and communicate such parameters to the MVS.
Training—Training typically requires that a user place an example of the object(s) to be analyzed by the MVS in its field of view, and then interact with the sensor as appropriate so that the sensor can learn necessary details of the object's visual appearance for subsequent analysis during production. Training generally requires a mechanism for displaying an image of the object to the human conducting the training procedure, and a mechanism for allowing the human trainer to choose points, lines, rectangles, and other shapes and HMI objects as appropriate.
Control—a HMI mechanism must be provided to allow users to control the operating state of the MVS, including placing it into operating states suitable for Installation, Configuration, Training, Diagnosis, Production, and Standby. A mechanism for other control operations such as MVS reset should be provided.
Monitoring—a HMI mechanism is typically provided to allow human users to monitor the operation of the MVS during production. Data to be provided to people for monitoring purposes includes images of the objects being analyzed, accept/reject statistics, numeric quality scores, and trends. Information about both the last object analyzed and the last object rejected is often useful.
It should be clear that the above-described Installation, Configuration, Training Control and Monitoring procedures are highly computer-centric, requiring a relatively fully configured PC with display and user interface to be deployed for each sensor or group of sensors in the work area. While the PC (laptop and desktop) is a powerful tool for providing a HMI, the cost of laptop and desktop PCs has not fallen appreciably over the past several years—due, in part, to their large component-count, complexity and continued improvement by manufacturers as a tradeoff against constant unit cost. Much of this increased computing power may be unnecessary for a standard MVS HMI. In addition, the weight and size of PCs and their displays has limited by their ergonomics. In other words, keyboards, mice, hard drives and displays take up space and cannot be overly miniaturized. There are also hidden costs to operating one or more PCs as HMIs. The PC's complexity means that it typically requires periodic intervention by costly Information Technology professionals to repair and upgrade operating system, networking and application software. Conversely, the size, and cost of machine vision sensors has continued to fall over the years, and they have been deployed at an increasing number of locations within production environments. A fully configured, current model PC may not provide the ideal HMI solution for all MVS applications.
In fact, there are many processes using a minimal number of trained vision procedures (for example a gear inspection process) where, once trained, the system may need little or no intervention by an user/operator for several weeks or months. It may be desirable to deploy such basic machine vision systems at a large number of locations around a production facility. Given the low cost and minimal intervention required for such systems it is desirable to provide an inexpensive HMI that enables easy installation, configuration, training and monitoring by a user/operator without requiring a bulky and expensive fixed-base PC that is permanently tied to the MVS. In this manner installation and operational costs can be significantly lowered. Alternatively, providing an interface on a remote fixed laptop or desktop PC that uses only a universal software application, such as a web browser, to interface with the sensor(s) would be quite desirable.
One possible candidate for a universal, portable HMI is the Personal Digital Assistant (PDA), also sometimes termed a “palmtop” computer. Within the past few years the PDA has become increasingly common, the most popular example being the Palm Computing® platform developed by 3Com Corporation, and sold by a variety of vendors including Palm, Inc. of Santa Clara, Calif. These devices sell for approximately one-tenth the cost of a PC, and around one-twentieth the size and weight. PDAs are capable in principle of machine vision HMI, since they have the four necessary elements—a display, a human input device, an interface that could be connected to an MVS, and available HMI software building blocks.
PDAs present significant obstacles to their use for machine vision HMI, however. The display is quite limited in resolution, both spatially (i.e. in pixels) and in intensity (i.e. number of gray levels). Both the spatial and intensity resolution are far less than that used by even the least capable machine vision sensor. PDA communication interfaces that could be used for the MVS (serial and infrared) are similarly limited in performance, operating at much less than the bandwidth needed for full real-time image display. The processing power of a typical PDA may be around 100 times less than a typical PC, and generally much less than an MVS, which may limit the ability to use the processor to overcome other limitations. For example, using sophisticated image compression algorithms to overcome the interface bandwidth limits may not be practical. Finally, installing and maintaining the right MVS software on a PDA might incur hidden costs. Hence, it is highly desirable to provide a manner to overcome these obstacles, and/or provide an effective HMI that can be applied to a more-generalized PC software and hardware architecture.